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1.  Introduction 


Classification  using  the  Man-portable  Vector  (MPV) 
advanced  electromagnetic  sensor  was  demonstrated  at  the 
former  Camp  George  West  Artillery  Range,  CO  in  2012. 

This  report  summarizes  the  results  of  that  demonstration. 

The  document  Implementing  Classification  on  Munitions 
Response  Sites  (Ref.  1)  provides  practical  information  for 
deciding  whether  classification  is  appropriate  to  a 
particular  site  and  how  it  is  best  implemented. 

Classification  is  motivated  by  the  need  to  perform 
munitions  response  more  cost-effectively  so  that  limited 
clean  up  dollars  can  be  used  to  reduce  real  risk  on 
munitions -contaminated  sites  sooner.  The  estimated 
liability  in  the  FY 1 0  Defense  Environmental  Programs 
Report  to  Congress  for  Munitions  Response  is  $15. 2B. 

(Ref.  2)  The  bulk  of  this  liability  is  $10. OB  for  the  1703 
sites  identified  in  the  Formerly  Used  Defense  Sites 
(FUDS)  program  and  $4.4B  for  the  2433  sites  identified 
on  Active  Installations.  The  remaining  $0.8B  is  in  Base 
Realignment  and  Closure  (BRAC).  The  estimated 
completion  dates  for  many  sites,  particularly  in  the  FUDS 
program,  are  decades  out  if  they  are  to  be  cleaned  up  at 
planned  funding  levels  using  current  practice. 

When  a  munitions  response  site  is  cleaned  up,  in  most 
cases,  it  is  mapped  with  a  geophysical  sensor  and  the 
locations  of  all  detectable  signals  are  excavated.  Geophysical  sensors  detect  metal  and,  therefore, 
many  of  the  detections  do  not  correspond  to  munitions,  but  rather  to  harmless  metallic  objects. 

Field  experience  indicates  that  95-99%  or  more  of  objects  are  found  to  be  nonhazardous.  Current 
technology  does  not  provide  a  means  to  discriminate  between  munitions  and  other  items,  termed 
“clutter.”  As  a  result,  most  of  the  costs  to  remediate  a  munitions-contaminated  site  using  current 
methods  are  spent  on  excavating  targets  that  pose  no  threat. 

Classification  is  a  process  used  to  make  a  decision  about  the  likely  origin  of  a  signal.  In  the  case  of 
munitions  response,  high-quality  geophysical  data  can  be  interpreted  with  physics-based  models  to 
estimate  parameters  that  are  related  to  the  physical  attributes  of  the  object  that  resulted  in  the  signal, 
such  as  its  physical  size  and  aspect  ratio.  The  values  of  these  parameters  may  then  be  used  to 
determine  whether  the  signal  arose  from  a  munition  or  harmless  clutter.  With  reliable  classification, 
only  the  munitions  need  to  be  removed  from  the  site. 

The  Strategic  Environmental  Research  and  Development  Program  (SERDP)  and  the  Environmental 
Security  Technology  Certification  Program  (ESTCP)  have  supported  the  development  of  purpose- 


Camp  George  W est  Artillery  Rnnge, 
CO  —only  one  munition  expected,  open 
access  with  challenging  slopes,  minimal 
geologic  interfierence,  medium  anomaly 
density 

Munitions  —  75-mm  projectiles 

Results  —  Roth  dynamic  and  cued  data 
were  collected  with  the  Man-portable 
Vector  Sensor  (MPV).  Anomalies 
were  identifired firom  the  dynamic  survey 
data  and  revisited  for  cued  interrogation. 
The  cued  data  were  used  to  successfully 
classify  all  of  the  targets  of  interest  and 
eliminate  more  than  90%  of  the  clutter. 
One  analyst  used  the  dynamic  data  to 
classify  the  targets.  Kesults  were,  in 
general,  good  but  one  TOI  was  left  just 
past  the  analyst’s  threshold. 
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built  advanced  electromagnetic  sensors  and  associated  analysis  methods  for  classification.  Following 
the  successful  demonstration  of  classification  methods  in  controlled  test  environments,  ESTCP 
initiated  a  Classification  Pilot  Program  to  validate  the  application  in  real-world  conditions.  The  goal 
of  the  program  is  to  demonstrate  that  classification  decisions  can  be  made  using  an  explicit 
approach,  based  on  principled  analysis  that  is  transparent  and  reproducible.  The  demonstrations  are 
planned  and  conducted  in  cooperation  with  regulators  and  program  managers  in  the  Services. 

The  physics  governing  the  electromagnetic  response  of  a  metal  object  is  well  understood  and 
predictable.  Data  collected  with  these  sensors  contain  the  same  information  content  on  any  site  and 
demonstrations  to  date  have  confirmed  that  classification  works  predictably.  Nevertheless, 
demonstrations  will  be  required  at  a  number  of  sites  to  represent  the  wide  variability  in  munitions 
types,  target  densities,  terrain,  vegetation,  geology,  land  use  history,  future  land  use,  and  other  site 
characteristics  that  will  affect  the  applicability  of  classification  and  to  establish  cost  effectiveness  and 
implementabiUty.  The  demonstrations  also  present  an  opportunity  to  work  out  standard  operating 
procedures  and  establish  quality  control  (QC)  measures.  Prior  demonstrations  have  been  conducted 
at  a  number  of  sites  across  the  country;  details  about  past  and  ongoing  demonstrations  can  be  found 
on  the  SERDP-ESTCP  web  site  at  http:/ / serdp-estcp.org/Featured-Initiatives/Munitions- 
Response-Initiatives  /  Classification-Applied-to-Munitions-Response. 

The  demonstration  at  Camp  George  West  continues  the  tests  of  smaller  man-portable  advanced  EM 
sensors  intended  for  use  where  terrain  or  vegetation  demand  a  more  maneuverable  option.  These 
sensors  are  less  mature  and  field  procedures  are  not  as  well  established  compared  to  the 
MetalMapper.  For  this  reason,  it  is  premature  to  attempt  a  cost  analysis  of  the  use  of  this  sensor. 
Data  were  analyzed  by  experienced  teams  from  the  developers  of  the  classification  methods.  Table 
1  shows  the  participants  and  their  roles  in  the  Camp  George  West  demonstration. 


Table  1.  Participants  in  the  MPV  Demonstration  at  Camp  George  West 


Task 

Performer(s) 

Task 

Performer(s) 

Site  Preparation 

Parsons 

MPV  Data 

Analysis 

Dartmouth  College 

SAIC 

Sky  Research,  Vancouver* 

MPV  Data 
Collection 

Sky  Research,  Vancouver* 

Intmsive 

Investigation 

Parsons 

Scoring 

Institute  for  Defense 
Analyses 

*Sky  Research,  Vancouver  personnel  are  now  affi 

iated  with  Black  Tusk  Geophysics 
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2.  Camp  George  West  Demonstration  Flow 

The  sequence  of  the  demonstration  is  outlined  in  the  flow  chart  in  Figure  2-1. 


Figure  2-1.  Flow  chart  outlining  steps  in  the  demonstration  at  Camp  George  West.  Blue  boxes  are 
tasks  performed  by  ESTCP.  Others  are  tasks  performed  by  contractors. 

Prior  to  the  beginning  of  data  collection,  an  instrument  veriflcation  strip  (IVS)  was  installed  and  the 
site  was  seeded  with  inert  munitions  and  medium  industry 
standard  objects  (ISOs),  2-in  nominal  x  8-in  pipe  nipples. 

(Ref.  3)  The  data  collection  team  visited  the  IVS  twice  daily 
to  verify  equipment  function  at  the  start  and  end  of  each 
day.  Since  there  are  few  native  unexploded  ordnance 
(UXO)  on  any  munitions  response  site,  the  seeds  provided 
sufficient  targets  of  interest  (TOl)  to  allow  a  statistically 
defensible  determination  of  the  correct  classiflcation  of 
TOl. 

The  MPV  sensor  (Ref.  4)  was  used  in  both  dynamic  and 
cued  mode  for  this  demonstration.  A  dynamic  survey  was 
conducted  to  detect  anomalies.  Each  anomaly  was  then 


Targets  of  Interest  (TOl)  are  all  objects 
that  must  be  removed  from  the  site. 
Typically  the  TOl  will  include  all 
known  or  suspected  munitions  types,  any 
other  unexpected  munitions,  munitions 
parts  such  as  fuges  that  present  an 
explosive  ha^rd,  and  all  seeded  items. 
When  classification  is  applied  to  a  site, 
the  local project  team  will  decide  what 
items  constitute  TOL 
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revisited  in  cued  mode.  All  detected  targets  were  dug  up  to  provide  complete  ground  tmth  for  the 
purposes  of  determining  performance.  The  UXO  technicians  photographed  each  item  that  was  dug 
and  recorded  its  location,  depth,  and  description. 

The  dynamic  and  cued  geophysical  data  were  passed  to  the  data  analysis  teams.  A  complete 
overview  of  the  analysis  procedures  can  be  found  in  Ref  1.  Briefly,  the  analysts  used  methods  based 
on  the  dipole  model  to  estimate  target  parameters.  Analysts  were  offered  training  data  from  test  pit 
measurements  and  the  opportunity  to  request  additional  training  data  from  the  recovered  targets,  as 
though  they  were  doing  a  limited  number  of  sample  digs.  These  data  were  used  to  set  classifier  rules 
-  the  decisions  that  separate  the  anomalies  into  TOI  and  non-TOI.  The  classifiers  were  then 
applied  to  aU  of  the  targets  that  remained  blind  for  each  demonstrator.  Since  training  data  was  by 
request,  the  blind  target  set  was  different  for  each  demonstration. 


The  product  required  from  each  analyst  was  a  ranked  anomaly  Ust;  an  example  is  shown  in  Figure 
2-2.  One  and  only  one  judgment  is  required  for  each  entry  on  the  anomaly  Ust.  The  first  items 
below  the  training  data  on  each  anomaly  list  are  those  targets  for  which  reliable  parameters  cannot 
be  extracted  and  therefore  must  be  dug.  Next  are  those  items  which  the  analyst  is  the  most 
confident  are  TOI.  These  items  are  ranked  according  to  decreasing  UkeUhood  that  the  item  is  a 
TOI.  Any  items  which  the  analyst  was  able  to  analyze  but  was  not  able  to  make  a  classification 
decision  on  at  this  time  were  placed  next  on  the  anomaly  Ust.  Last  are  aU  those  items  that  the  analyst 
was  confident  are  not  TOI  ranked  by  their  UkeUhood.  The  analysts  were  required  to  specify  a  size 
estimate  for  any  item  marked  to  be  dug.  This  initial  Ust  is  shown  in  the  left  panel  of  Figure  2-2. 


Initial  Ranked  Anomaly  List 


First 

Pass 


Final  Ranked 
Anomaly  List 


Figure  2-2.  Example  initial  and  final  Ranked  Anomaly  Lists.  A  detailed  description  is  in  the  text. 
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The  seeds  were  divided  into  QC  seeds  and  blind  seeds.  When  analysts  submitted  their  initial 
prioritized  lists,  the  QC  seeds  were  used  to  provide  feedback  if  seed  targets  were  missed.  Analysts 
were  also  provided  with  the  ground  truth  information  on  all  anomalies  in  the  red  part  of  their  lists 
and  any  requested  anomalies  in  the  yellow  part.  This  is  signified  by  the  threshold  on  the  left  list  in 
Figure  2-2.  Based  on  this  information,  the  analysts  were  then  allowed  to  revisit  their  rankings  and 
assignments  for  all  items  that  were  stiU  blind  until  they  were  satisfied  that  the  best  possible 
classification  had  been  achieved. 

In  the  final  Ust,  shown  in  the  right  panel  of  Figure  2-2,  the  analyst  is  required  to  provide  a  threshold 
that  corresponds  to  the  division  between  those  items  recommend  for  digging  and  those  that  can 
safely  remain  in  the  ground.  That  is,  every  item  on  the  Ust  is  marked  to  be  dug  or  not  with  a 
threshold  separating  the  two  categories.  The  final  prioritized  anomaly  lists  submitted  in  this 
demonstration  were  scored  against  the  emplaced  bUnd  seeds  and  recovered  targets  by  IDA. 

3.  Site  Description  and  Preparation 

The  135-acre  Camp  George  West  Artillery  Range  Munitions  Response  Site  (MRS)  was  used  by  the 
Colorado  Army  National  Guard  for  artiUery  training  from  1930  to  1945  as  an  impact  area  for  75- 
mm  high  explosive  (HE)  projectiles.  (Ref.  5)  Other  portions  of  the  artillery  range,  including  firing 
points,  forward  observer  position(s),  and  surface  danger  zones  are  not  precisely  known.  The  MRS 
(impact  area)  is  currently  owned  by  the  city  of  Lakewood. 

In  order  to  minimize  damage  to  the  recreation  area,  personnel  from  the  City  of  Lakewood  and  the 
ESTCP  Program  Office  settled  on  a  2-acre  demonstration  area  for  this  site.  The  specific  2  acres 
were  chosen  based  on  anomaly  densities  interpolated  from  an  initial  transect  survey.  An  aerial 
photo  of  the  site  is  shown  in  Figure  3-1  with  the  2-acre  demonstration  area  marked  in  red. 

All  visible  metal  objects  were  removed  from  the  surface  at  the  site.  First  order  reference  points  were 
instaUed  by  a  registered  surveyor  for  geolocation  reference.  An  area  free  of  contamination  was 
located  near  the  demonstration  area  to  estabUsh  an  IVS  used  for  daily  verification  of  proper  sensor 
operation  and  a  training  pit  to  coUect  sensor  data  for  algorithm  training. 

The  only  known  munition  at  this  site  is  75-mm  HE  projectiles.  The  analysts  were  provided 
information  about  the  historical  use  and  known  munition.  In  addition  to  this  munition, 
demonstrators  were  directed  that  any  unexpected  munitions  would  also  be  considered  TOl. 

The  final  demonstration  site  was  seeded  with  enough  TOl  to  ensure  statistical  vaUdity  on  measures 
of  classification  of  targets  of  interest.  The  TOl  seeds  are  Usted  in  Table  3-1.  The  seeds  included 
not  only  inert  75-mm  projectiles,  but  also  medium  industry  standard  objects  (ISOs).  (Ref.  3)  The 
ISOs  are  also  considered  TOl  and  expected  to  be  both  detected  and  correctly  classified. 
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Figure  3-1.  Aerial  photo  of  the  Camp  George  West  Artillery  Range  showing  the  demonstration  area 


For  safety,  seeds  were  emplaced  using  standard  anomaly  avoidance  procedures.  No  attempt  was 
made  to  separate  the  seeds  from  the  surrounding  clutter.  For  realism,  the  emplacement  teams  were 
instructed  to  replace  any  metal  dug  up  during  emplacement  back  in  the  hole  with  the  seeded  object. 


Table  3-1.  Seeds  Emplaced  for  the  Camp  George  West  demonstration 


Item 

Number 

Depth  Range  (cm)* 

Industry  Standard  Object  -  Medium 

10 

15-45 

75-mm  projectile 

30 

15-46 

*Depths  are  to  the  center  of  the  object. 
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4.  MPV  Data  Collection 


The  MPV  is  a  time-domain,  electromagnetic  induction  (EMI)  sensor  composed  of  a  single 
transmitter  coil  and  an  array  of  five  receiver  units  that  measure  aU  three  components  of  the  EM  field 
as  shown  in  Figure  4-1.  The  MPV  sensor  head  for  this  demonstration  comprised  a  50-centimenter 
(cm)  diameter  circular  loop  transmitter  coiled  around  a  disk  that  intermittently  illuminates  the 
subsurface,  and  five  8-cm  multicomponent  receiver  units  (cubes)  that  measure  the  three  orthogonal 
components  of  the  transient  secondary  EM  field  decay. 


Figure  4-1.  Components  of  the  MPV.  Left  inset  shows  data  acquisition  (DAQ)  and  power  unit 
mounted  on  a  backpack  frame.  Right  inset  shows  the  sensor  head  from  above  with  cubes  numbered. 

MPV  data  were  collected  in  two  modes.  Survey,  or  dynamic,  data  are  collected  to  identify 
geophysical  anomalies.  These  full  coverage  data  are  collected  using  the  attached  GPS  antenna. 

After  anomalies  have  been  identified,  they  are  revisited  in  static,  or  cued,  mode  to  collect  more 
detailed  information  about  the  source  of  the  anomaly.  The  beacon  system  is  used  for  local 
positioning  in  this  mode. 

4.1  Dynamic  Survey 

The  dynamic  MPV  survey  was  carried  out  along  lanes  laid  out  across  the  site  by  sweeping  the  sensor 
from  side  to  side  across  the  lane.  (Ref.  6)  The  two-person  survey  crew  is  seen  in  Figure  4-2.  RTK 
GPS  is  used  for  geolocation  during  the  dynamic  survey.  Production  rates  averaged  0.7  acres  per  day. 

The  anomaly  selection  criterion  for  the  detection  survey  was  based  on  the  75-mm  projectiles 
expected  at  the  site  at  a  depth  of  two  feet.  This  depth  was  chosen  as  the  deepest  depth  to  which  a 
75-mm  could  be  reliably  detected. 
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Figure  4-2.  Dynamic  MPV  survey  at  Camp  George  West 

Since  the  MPV  is  a  hand-held  sensor,  the  height  of  the  sensor  head  above  the  ground  is  not  fixed  as 
would  be  the  case  for  a  sensor  mounted  on  wheels  or  skids.  This  complicates  the  calculation  of  a 
sensor  response  curve  that  would  normally  be  used  to  set  the  anomaly  selection  amplitude.  For  this 
demonstration,  the  anomaly  selection  threshold  was  based  on  a  Monte  Carlo  simulation  of  the  signal 
amplitude  expected  for  a  75-mm  projectile  at  60  cm  below  the  ground  surface.  For  each  of  the  2000 
realizations  in  the  simulation,  the  projectile  was  oriented  randomly,  the  sensor  head  varied  between 
5  and  10  cm  above  the  surface,  and  the  pitch  and  roll  of  the  sensor  varied  from  0  to  20  degrees. 

The  results  of  the  simulation  are  shown  in  Figure  4-3. 


'  simulations 
X  directly  under 
0  30  cm  to  the  side 

A  35  cm  in  front 
- threshold 


Figure  4-3.  Monte  Carlo  simulation  of  the  response  of  a  75-mm  projectile  at  60  cm  below  the  ground 
to  the  MPV.  The  dots  correspond  to  the  2000  conditions  simulated,  the  symbols  correspond  to  test 
stand  measurements  made  to  validate  the  simulation,  and  the  dashed  line  corresponds  to  the 

threshold  chosen  for  this  demonstration. 
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The  anomaly  selection  threshold  was  set  so  that  99%  of  the  75-mm  projectiles  at  60-cm  depth 
would  be  detected.  Of  course,  shallower  projectiles  would  be  detected  at  higher  Pq.  Using  this 
threshold,  there  were  567  exceedances  in  the  2-acre  demonstration  area.  Eighty  four  of  the 
threshold  exceedances  were  noise  spikes  that  were  rejected  as  anomalies  leaving  483  anomalies  for 
cued  re-investigation.  All  40  seed  items  were  included  on  this  list. 

4.2  Cued  Survey 

The  MPV  returned  to  each  of  the  483  anomalies  identified  from  the  dynamic  survey  for  cued  data 
collection.  In  this  mode,  MPV  initially  collects  data  in  a  five-point  pattern  -  one  measurement 
directly  over  the  anomaly  and  four  measurements  around  the  anomaly  location.  If  significant  signal 
is  observed  from  the  outer  receivers,  additional  points  are  collected.  Local  positioning  in  cued  mode 
is  provided  by  a  location  beacon.  Figure  4-4. 


Figure  4-4.  MPV  cued  data  collection  at  Camp  George  West.  The  location  beacon  used  for  local 
positioning  during  cued  operation  can  be  seen  on  the  right  side  of  the  photograph. 

Analysis  of  the  cued  data  showed  that  47  more  of  the  anomalies  were  due  to  noise  spikes  in  the 
survey  data  and  30  additional  buried  items  were  identified  from  anomalies  caused  by  multiple 
objects.  After  accounting  for  this,  a  total  of  466  items  were  submitted  for  analysis  and  subsequently 
intrusively  investigated.  The  data  collection  team  averaged  135  cued  anomalies  per  day.  (Ref.  6) 

5.  Intrusive  Investigation 

A  summary  of  the  results  of  the  intmsive  investigation  is  shown  in  Figure  5-1.  The  left  panel  in  the 
figure  shows  the  distribution  of  the  number  of  items  that  were  recovered  from  a  single  hole.  As 
expected  given  the  moderate  anomaly  density  at  this  site  (~250/ acre),  approximately  75%  of  the 
anomalies  were  caused  by  a  single  buried  item.  The  pie  chart  on  the  right  hand  side  of  Figure  5-1 
shows  the  identities  of  the  recovered  objects.  The  majority  of  the  recoveries  were  identified  as 
munitions  debris;  not  unexpected  for  an  impact  area.  Only  seventeen  items  were  identified  as 
“other  debris”  which  is  surprising  given  the  recreational  use  of  the  site. 
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I  I  Munitions  Debris 

I  I  QC  Seeds 

I  I  Other  Debris 

I  I  No  Contact 


Number  of  Items  Recovered  per  Anomaly 


Identity  of  Recovered  Items 


Figure  5-1.  Distribution  of  items  recovered  per  anomaly  (left  panel)  and  identity  of  recovered  items 

(right  panel) 

Nine  UXO  were  recovered  as  part  of  this  demonstration.  All  nine  of  these  items  were  disposed  of 
by  detonation  in  place  by  the  UXO  technicians. 

The  depth  distribution  of  the  recovered  items  is  given  by  the  blue  hne  in  Figure  5-2.  As  at  most 
munitions  sites,  the  vast  majority  of  items  were  recovered  in  the  top  10  to  20  cm.  The  depths  of  the 
recovered  UXO  are  indicated  by  the  stars  on  the  left  side  of  the  plot  and  the  depth  range  of  the 
seeds  by  the  grey  bars.  The  anomaly  selection  criterion  was  based  on  projectiles  at  60  cm,  far  deeper 
than  any  item  recovered  in  this  demonstration. 


Number  of  Recovered  Items 
0  20  40  60  80  100  120 


Figure  5-2.  Depth  distribution  of  recovered  items  from  Camp  George  West  (blue  line).  The  depths 
of  the  nine  UXO  are  marked  by  stars  and  the  depth  range  of  the  QC  seeds  are  shown  as  grey  bars. 
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The  nine  recovered  UXO  are  shown  in  Figure  5-3.  They  ranged  in  size  and  condition  from  nearly 
complete  75-mm  projectile  to  sections  on  the  order  of  one-half  to  one-third  of  a  complete  projectile. 


m 


^/onRLy-  Gw-‘K)52 


DePT^- 

l,E.W(5iTl<-  'Z.OcnvN 

75««  W/ GRAPE  SHOT 


Dig,  ryf^  -  ttPPEH 


B'/s-a  1 


Ar-.  ALY'GW-lfol 
-£  :&TH-U.'6ca 

'T.rCTij'g- 


Atr-  4w  /J271 


H-  <-A 

GTH-  (7,£a» 


/^Pf 


Figure  5-3.  Recovered  UXO  from  this  demonstration 


6.  Analysis  of  MPV  Data 

The  MPV  data  from  this  demonstration  were  analyzed  by  the  firms  that  are  developing  analysis 
methods  for  the  advanced  EMI  sensors.  AH  of  the  analysts  achieved  very  good  results;  a  sample  of 
the  analyses  will  be  presented  here.  A  complete  description  of  the  analysis  methods  used  and  results 
can  be  found  in  the  demonstration  reports  prepared  by  the  individual  analysts  (Refs.  6-9) 
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6.1  Analysis  of  Cued  Data 

An  analyst  from  Dartmouth  College  processed  the  cued  MPV  data  using  methods  developed  in 
house.  (Ref.  7)  The  results  of  this  analysis  are  shown  in  Figure  6-1.  The  colors  on  the  plot 
correspond  to  the  red  and  green  colors  in  the  final  ranked  anomaly  list  as  shown  in  Figure  2-2.  The 
red  are  the  items  the  analyst  classified  as  “high  likelihood  TOI”  and  the  green  are  those  the  analyst 
called  “high  likelihood  not  TOI.”  No  anomalies  were  classified  as  “unable  to  classify.” 
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Figure  6-1.  Results  of  a  Dartmouth  analysis  of  the  cued  MPV  data 

The  graph  plots  the  percent  of  the  TOI  correctly  classified  on  the  vertical  axis  and  the  number  of 
clutter  items  on  the  horizontal  axis.  The  offset  from  zero  in  the  starting  point  reflects  the  training 
data  that  the  analyst  requested.  Any  anomalies  classified  as  “can’t  extract  reliable  parameters”  would 
be  represented  by  an  initial  black  line;  no  anomalies  were  in  this  category  for  this  analysis.  The  blue 
dot  represents  the  threshold  selected  by  the  analyst  and  the  orange  dot  shows  the  point  on  the 
ranked  anomaly  list  where  100%  of  the  TOI  are  captured.  Ideally,  a  classifier  would  correctly 
identify  all  targets  of  interest  in  the  red  with  zero  clutter  and  all  of  the  clutter  would  be  in  the  green. 
In  this  case,  the  red  part  of  the  curve  would  go  straight  up  to  1 00%  and  the  green  part  of  the  curve 
would  mn  straight  across  the  top  axis.  Success  in  these  demonstrations  was  defined  by  eliminating 
the  maximum  amount  of  clutter  while  correcdy  identifying  all  of  the  TOI. 

In  this  demonstration,  there  were  409  total  clutter  items  as  determined  from  the  ground  truth.  The 
Dartmouth  analyst  was  able  to  correctly  identify  all  but  30  of  these  items  at  his  threshold,  for  a 
possible  savings  of  more  than  90%  of  the  digs. 

An  analyst  from  Sky  Research  (now  affiliated  with  Black  Tusk  Geophysics)  achieved  similar  results. 
Figure  6-2.  This  analyst  used  the  27  training  digs  requested  (8  of  which  were  TOI  and  19  clutter)  to 
verify  that  there  were  no  unexpected  small  ordnance  present  on  the  site  and  then  used  a  restricted 
library  to  identify  all  the  remaining  TOI  while  only  digging  three  more  clutter  items.  This  illustrates 
the  performance  possible  at  a  site  where  the  number  of  potential  munitions  types  is  limited. 
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Figure  6-2.  Results  of  the  Sky  Research  analysis  of  the  cued  MPV  data 
6.2  Analysis  of  Dynamic  Data 

Another  analyst  from  Sky  Research  (also  currently  affiliated  with  Black  Tusk  Geophysics)  used  only 
dynamic  MPV  data  to  extract  target  parameters  and  prepare  a  ranked  anomaly  Ust,  Figure  6-3.  This 
analyst  also  achieved  good  results  although  the  threshold  chosen  left  one  UXO  not  dug  (five  items 
on  the  list  after  the  threshold). 

As  expected,  use  of  the  dynamic  data  results  in  a  ROC  curve  that  is  not  as  sharp  as  that  achieved 
using  the  cued  data.  The  extra  clutter  digs  required,  however,  are  more  than  paid  for  by  not  having 
to  collect  cued  data.  As  data  collection  teams  and  analysts  become  more  familiar  with  dynamic  data, 
it  will  be  possible  to  draw  wider  conclusions  about  the  usefulness  of  dynamic  data  alone. 
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Figure  6-3.  Results  of  the  Sky  analysis  of  the  dynamic  MPV  data 
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7.  Conclusions 


The  Man-portable  Vector  Sensor  was  used  on  the  former  Camp  George  West  ArtiUery  Range  in 
both  dynamic  and  cued  mode.  Anomalies  were  identified  from  the  dynamic  survey  data  using  a 
threshold  based  on  the  signal  expected  from  a  75-mm  projectile  at  60-cm  depth.  All  QC  seeds  were 
detected  using  this  threshold.  Cued  data  was  collected  in  a  five-point  pattern  over  466  anomalies  in 
the  2-acre  demonstration  area.  Analysis  of  these  cued  data  yielded  excellent  results.  AU  TOI  were 
correctly  identified  and  the  best  analyst  was  able  to  leave  nearly  95%  of  the  clutter  in  the  ground. 
One  analyst  used  the  dynamic  data  only  to  classify  the  anomalies;  the  results  were  good  but  the  last 
TOI  on  the  list  was  five  items  after  the  analyst’s  stop-dig  threshold. 

The  Camp  George  West  site  is  an  excellent  site  for  classification  using  advanced  EMI  sensors.  The 
single,  relatively  large  munition  expected  allows  the  analysts  to  efficiently  screen  out  fragments  and 
other  debris.  The  best  performing  analyst  used  the  training  digs  to  confirm  that  no  smaller 
munitions  were  present  in  the  demonstration  area  and  then  used  a  restricted  library  to  very  precisely 
identify  the  TOI. 

The  MPV  sensor  was  successful  in  both  dynamic  and  survey  mode  on  the  slopes  encountered  at  this 
site.  As  more  demonstrations  using  the  portable  sensors  and  dynamic  data  collection  are  performed, 
we  win  be  able  to  better  assess  the  range  of  sites  for  which  these  methods  are  applicable. 


8.  Acronyms 


BRAC 

CO 

EM 

EMI 

ESTCP 

FUDS 

GPS 

IMU 

ISO 

IVS 

MPV 

QC 

SERDP 

TOI 

UXO 


Base  Realignment  and  Closure 
Colorado 
Electromagnetic 
Electromagnetic  Induction 

Environmental  Security  Technology  Certification  Program 

Formerly  Used  Defense  Site 

Global  Positioning  System 

Inertial  Measurement  Unit 

Industry  Standard  Object 

Instmment  Verification  Strip 

Man-portable  Vector  [Sensor] 

Quality  Control 

Strategic  Environmental  Research  and  Development  Program 
Target  of  Interest 
Unexploded  Ordnance 
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